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ABSTRACT 
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and iimffoved systems for viewing a sporting event In 
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forms of infonnation having characteristics dependent on a 
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VISION SYSTEM FOR VIEWING A and is combined witfi the image of the real scene. The 

SPORTING EVENT infonnation which relates to the scene can be embodied in 

many foims. but is generally related to certain spatial 

This {plication is a continuation-in-^art of a previously features of the scene. It is critical to note that the information 

filed application having a Ser. No. of 087119360 filed on 3 relates to a particular scene, and is merged with a real 

Sep. 10, 1993, whidi is currently pending. The instant electronic image of die scene by a oonq>uter sudi that the 

invention is generally concerned witii an elec^o-optic vision pcrspecdve of the displayed information corresponds to ttie 

system which exploits knowledge of position and attimde to uscr*s perspective and is aligned to the scene being 

display information in a perspective unique to that position addressed. 

and attitude, and is specifically concerned with a vision lO Some types of infonnation that may be merged with an 
system for use in a sporting event to view various features in^e of a real scene being address indude historic, 
associated with the game from a users perspective. This st atistical, symbolic, abstract symbolic, a rtion tracks, histo - 
disclosore relies on the teachings of the above-identified gram of action trades^ computer gr^lucal user interface 
q>plication therefore the entire content of that application is devices. Each of these types, can be combined with an image 
incorporated herein by reference. As prefecced embodiiiients 15 of a real scene in a spa^ally dq)endent fashion to contribute 
are directed to the specific game of baseball and many to further understanding of the scene being addressed. We 
references are made thereto, the ^ules of Baseball** as set caU the con^site image that has been augmented with 
foitii by the Commissioner of Baseball are assumed to be information from memory: "Augmented Reality". To 
well known by those familiar with the art It should be present a basic understanding of the various information 
understood that all of the herein referenced materials are 20 types, a single example of some of them follows; a corn- 
provide considerable definidon of elements and features of plete and detailed explanation appears in the detailed 
the present invention, description portion of this disclosure. Images of real scenes 

can t)e combined with recordings of images and image series 

BACKGROUND OF THE INVENTION from past times which share same or similar spatial features 

Cunentiy, electronic devices to aid in the comprehension " of present scenes. Arcal scene can be improved by inserting 

and understanding of the game of basebafl are being tested information relating to spatial f^^tures of the 

in stadium environments. Particularly, a device caUed scene. Symbols to rej^sent otherwise mvisible image fea- 

"SpcrtsIVax" made by MottKXrfa is being tested at the of a scene can be added m appropriate locations. 

Toronto Blue Jays games. Users can program an audible Similariy, symb(ds to represent abstract geometrical featores 

alertsfliatindicatetheendofaninning,arunscored,ahome ^ such as property hnes or planar boundanes can also be added 

ran, and when a rally is taking pLacc. If an opposing player ^o*"^ ^ iUustrated by highU^ting a 

belts one out a crying sound is heard. Because data is ^ ^ "^^^ ^ movmg object has 

entered from the press box and transmitted to users after a tmvdcd. When an otgect tates a similar path m a scries of 

two minute delay, the usefulness is limited for spectators ^^"^ motions, a history d those paths can be presented 

present at the game. The technology is crude and has been tog^er to fOTn a histogram type plot As wifli many 

adapted from the pager technologies known in messaging con^Hiter systems, user inputs can be via a congniter gr^hi- 

arts. Whereas baseball is a fascinating sport wim great ^ "^er interface means such as *Vindows^ '"pcAnt and 

opportunities of pleasure for spectators in attendance at the ^<^'^ and drag". A special form of window we caU 

game, systems of the present invention are configured to ^ "P^^r is used in some versions of the invenUon. Panels 

enhance visual aspects of the game. ^ ^« ^^ser interface and are sometimes related to 

spatial features of a scenes 

SUMMARY OF THE INVENTION Although nmy s pwting eventsJ iave a large spectatOT 

n^^^. «f « ^.ir^ particq>ation, the present disclosure is directed to the par- 

Comes now, an invention of a vision system includmg S^f^atyj^ of baseball. It wiU be easfly appreciated tot 

S is Fes^S'i^oJer pe^^ve and alfg^l^e ^ ^^'^ Tl^'J invention A 

reS^cenfas the scene 2w^w2from the lociion of the J"^"^ can be had wi& reference to the 

userDevicesoftheinventionbeiogprimarilycharacterized ^ embodiments and the 

by having a camera for coUecSnimage: a computer, a 50 append^dmwii^-nieseembodin^ 

^ st<^ which is respon^o a Son detaSg ^^^^ ^ T^T^'^'J^^ "^J'^^l 1^ 

J - • 1™— & ways possible to benefit fr«n the mvention. Therefore, there 

means and an attitude determuune means, and a di^lav for ' . , ^. . . ^ . ^ ^ 

luj^aua auw oil cuuiu^ u^auimmg " «i.i^«ijf euibodiments that do not deviate firom the scope 

presenting an augmented inoage to a user. J ^r: TT^^~; .7^ .1^ ^ 7^ a1 
6 &— *~e> spmi of the inventiOD as set forth by the claims, but do 

A camera can be used to f<Hm an dcctromc maagc of a ^ ^ cxanmles. 

particular scene of interest, the image having the identical 55 ^ ^ ^ ^ *v 1 „ 

^pective as the scene would a^ to h^e to a person Wherefore the ^nands are high ^invennon answers 

the scene without a visl^d. Using ima^pr<v ^ ^^^^^ ^J^- 

cessing routines, a con^ter can be used to manipulate the P^^"^ ^ invention to provide a vision 

electronic image before it is j^cscntcd at a dispUy. The system for augmentmg real miagcs. 
original image may be altered according to many known 60 It is also an object of the invention to provide a vision 
technologies such as color enhancement, edge system operable for combining computer generated images 
improvements, anti- vibration schemes, etc., but more with real images where the computer generated images are 
importantly, the original image may be augmented with presented in a perspective which corresponds to the user's 
information front a data store whCTe the information is perspective of a scene being addressed and further being 
particular to the scene being addressed as defined by the 65 aligned to the scene being addressed, 
position and attitude of the vision system- Information is It is a furttier object to provide a vision system which 
arranged by the con^uter to i^pear in the proper perspective combines the electronic image from a camera with inf or- 
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mation recalled from a data store where the infcnnation 
from the data stOTe is particular to the position and attitude 
of the camera. 

a is another primary object of the invention to provide a 
vision system for use in sporting events such as basebalL 

It is a further object of the invention to provide a vision 
system for viewing a live baseball game where images from 
the game are augmented with a computer. 

To arrive at these and other objects, features and 
advantages, devices having: 
an electr o-optic vision .system for viewing a sporting 
' event comprising: a c amera for produciQ ga n electronic 
image of a scene being addressed, position aMamEidc 
determining means for detcmiining the position and 
attitude of the camera, a data store responsive to the 
ou^uts of said position and attitude d^ermining means 
and having information stored therein, computer oper- 
able for combining the stored information with the 
electronic image to produce an augmented image, and 
an electro-optic display operable for displaying the 
augmented im^e; 
and methods having the steps of: 
producing an electronic image of a real scene with a 
imaging device, determining the position and attitude 
of the imaging device, recalling infomiation from a 
data store which depends on the position and attitude of 
the imaging device, augmenting the electronic image 
with the information from the data store, 
are provided. 

BRIEF DESCRIPnON OF THE DRAWINGS 

The features,, aspects, and advantages of the present 
invention will became better understood with regard to the 
following description, appended claims and drawings 
where: 

FIG. 1 is scenario showing baseball fans viewing a game 
through systems of the invention; 
FIG. 2 is a dose-up showing the augmented images of 

no. 1; 

FIG. 3 is an image of a baseball field and two coir^site 
images as viewed by two users; 

FIG. 4 is a block diagram of a version of the invention; 

FIG. 5 shows an image x>f a real scene and a computer 
generated image comMned to form a composite image; 

FIGS. 6, 7, and 8 show a simple geometry appearing in 
three different peE^)ectives; 

FIGS. 9, 10, and U *ow a d^ailed scene as it relates to 
baseball in three different perspectives; 

FIG. 12 shows an image of a player as recorded by a 
camera and a conqputcr generated gr^)hic superiiiq>osed 
thereon; 

HG. 13 shows a "Panel*" placed in the field of play by the 
computer; 

FIGS. 14A and 14B show abstract graphic data superim- 
posed onto real field images; 
FIG. 15 shows ball tracks; 

FIG. 16 shows a flow chart hieiarchy for a version of tiie 
invention; 

FIGS, 17-19 arc composite images of the invention; 

FIGS. 20-97 are flow diagrams of a prcfencd embodi- 
ment. Each figure after 20 is a subset thereof. The titles of 
each figure indicate a level within the flow diagram 
hierarchy. 
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DETAILED DESCRIPTION AND PREFERRED 
EMBODIMENTS OF THE INVENTION 
The baseball system is a dynamic, innovative system that 
allows a fan to observe things they otherwise couldn't see, 

5 understand things they mi£^ not otherwise oonqzrehend, 
and greatly enrich the game experience. The system exploits 
data related to the user's position and orientation to generate 
grq>hics that are combined with real images derived directly 
from a real world view. Using this system, the user would be 

10 able to see and, in some cases, interact with graphical 
r^esentations that allow the user to see things that arc not 
there or are obstructed. The system may have various 
elements in communication with each other whac each 
dement may be located remotely from one or more of the 

1^ other elements. An important element of the system is a hand 
held instrument which the user interacts with is sometimes 
herein called a •'Btighteyes'' unit or "hand held unit". 
Similar in size and weight to an electronic camcwdcr, it is 
generally in communication with remote computers and also 

20 has some conqniting facilities therein. HG. 1 shows two 
fans 11 in a similar location within the stadium but looking 
in di£ferent directions. The scene that a first fan might see is 
an outfielder throwing a ball 12. The scene tiie second fan 
may be viewing is the play at second base 13. It is fimda- 

25 mental to the invention that the system recognize what scene 
is being addressed, lb acoomplidi this» the hand held device 
14 has a reference viewing axis associated with it The 
pointing direction of this viewing axis of the first fan's unit 
21 is pointing in a different direction than the viewing axis 

30 of the second fan's unit The unit has a facilities to determine 
the pointing direction of the unit The position of the fans 
within the stadium is also measurable. Having position and 
attitude mcasurcments, a cowpntcr can dctennine what 
scene is being addressed by each fan. The computer can then 

3^ respcHid by augmenting an electronic image made with a 
camera with information generated by a conqMiter in accor- 
dance with the position and attitude measurements. 

SYSTEM HARDWARE CONHGURATIONS 

40 For basebali or similar apjtoations, a "Brighteyes" unit 
14 is supported by data services that are accessed by wireless 
means. Suitable radios, with repeaters If required, are com- 
mercially available. The radio provides communication 
between the Brigihteyes unit and larger con^utcr systems 

45 having larger con^Hitmg and memory capacity. There could 
be a large number of users present at the baseball park and 
that diese usees could be making simultaneous use of 
Brighteyes systems. To support this large number of usas, 
the radio commiiniration system may operate by hroadcast- 

50 ing some information. We divide the computer into sub- 
systems, based on function. A broadcasting sub-system is 
herein called an "Action Data System 46". Certain con^utcr 
requests and services would also be availaHc on a single 
user request basis. Single user requests are handled by a 

53 'Ttensaction Data System 48" 

Therefore, in preferred versions there are three con^Hiter 
sub-systems: a first which is a battery opaated Brighteyes 
unit held by a user, a second which is called an Action Data 
System 46, a oon^uter sub-system required to track and 

60 broadcast the position of the ball and all the players, 
managers, unqiiies and coaches involved in the play to die 
Brighteyes units; and a third which is called a Transaction 
Data System 48, a concqjutcr sub-system that provides 
statistics, information and other services on a broadcast or 

65 request ba^s to the Brighteyes units. Each of these computer 
sub-systems may be in communication with one or eadi of 
the others. 
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BRIGHTEYES HAND HELD UNITS triangulaUon to two points whose position is accurately 

A BestversionsofthehandheldbatteryopcratcdBrightcycs . |aiowntothcBrightcy«umtis<ii5(«^k^ 
^ unit contain: an input lens as_scm bly 41 ^ auto foSis iid ^ versions. In the event tot a Bn^ ^ imiU 

zoomandacoiSrcD44^Jh^ t^^'^^^^^;^^^^^^^ 

an attitodesenstog system 43 that provides information on 5 ^^E^^^^^^^^^pSST 

the pointing direction of the Bright^es unit; us^utteys ^^^^^^^^^^STSS Kt?^^ 

whiebytte user of the Brightcyw unit may int^-v^^ and was registered with &e stadium, the user o^ 

S^Sormatioo to the S^oway^o Whereby the ^if^ ^ ^f^^^^l^^f^ 

Brighteyes un it mav interact with and receive infgm ation and inake use of aU the services d^ari^ 

froS^aHffgive^Siito^ lO vi^t Smulariy, Bnghteyes units eqmpped 

Bii^Et^^SSrS'^r^f^^ crystal displays widi with addmonal opuonal equipment such as the capahflity to 

We^g lens assemblies such that the output images may be record and play badi video infonnation^ 

observ^ by the usff of the Brighteyes unit; a computer that spodiadio makmg use of a TV sub^^ 

has access to: the video streaTftom the electioTc camera (^-g- TVTl) could qx^rate at the stadimn or ball park 

assembly; data from the digital radio; k^yed input from the 15 ^/fy,?^ ^ Brighteyes um^ "J^^^If;^ 

user, at^de information from the attimde sensing system wito a suitab^ digital ra^ and wer^^^^ 

and position data. The unit may be equipped ^^Sl^l^dio ^ f^™ '''t^'' "^'"^ ^"""^"^ 

in/out system enabling speech processing to input spoken requested and provided. 

hif<OTnation to the Brighteyes conqniter. This may be in "^^^ the random access mmory of the Brighteyes umt, 

addition to or in substitution for odier means for infonnation 20 wouldbe die operating software of the Brighteyes conqnitcr 

input witfiin the prcforcd cmbodimcait The conq)utcx in the uni* a variety of data variables defined below. (Many 

Brighteyes unit also has access to the lens assembly for the . variables would also be stored such as: die control 

purpose of sensing die zoom and focus position of the lens Policy for the radio, the lightmg threshold to enable blanlang 

and for the farther purpose of commanding the lens to zoom of the flood li^ts, user schscted information), 

and focus as required by the control program running on the 25 ^*^thia the random access wcmar^ of the Brighteyes unit 

Brighteyes coin^juter and in response to requests from die would be a digital representation of die stadium ("Stadium 

user. The computer within die Bri^yes unit 45 can input Model") stored in such a way diat die X, Y, 2 positions and 

images from the real world from die lens assembly and identitiesof every significant non-moving object and surface 

augment, delete, and substitute infcmnation in graphical can be determined by software widun the Brigjiteyes unit 

ftirm before presenting the augmented conqjositc images to 30 accessing die digital representation. Additionally die digital 

die user at die display. representation would provide inf onnation enabling physical 

Q The conqmtcx within die Brighteyes unit 45 contams a s^^^ces sudi as die waDs and playing surfaccto be dMcr- 

3 small amount of non-volatile read only storage in which a entiated from open ^laccs such as «re sky. This Stadium 

unique identification number is stored during manufacture. Model could be downloaded to die Bri^teyes umts as 

Also widiintiiis read cmly storage is a control program ttiat 35 poiodicaUy broadcast from die Transaction Data System 48 

when accessed by the computer at power up causes die or it could be stored between games widiin die non-volatile 

Brighteyes unit to initialize and commence operation. The read write sta^e if fitted. 

conq>utei widiin die Brighteyes unit 45 also has random Widiin die RAM of die Mghteyes unit would be die 
access memory and, read-write non-vdatae storage (for position in teims of X, Y; Z of die ball in play and each 
cxan^dc battery backed up CMOS RAM or a hard-disk 40 player, manager, coach and un^e taking part in die current 
drive) for die storage of software and for die storage of play. We refer to diese as "Action Data Hements''. In die 
information b^een games. For purposes of cost, si2e and game of baseball dierc are generally 21 Action Data Elc- 
reduction in power consun^on. it would be possible to nicnts at any one time (sec below). The Action Data Ele- 
dispense witii diis read-write non volatile storage by increas- mcnts position and movements wwild be broadcast to the 
ing the amount of software within die read-wdy storage to 45 Brighteyes units by the Action Data System 46. Each Action 
include the capability to initiate data transfer over die radio Data Element would have a unique identifier to enable 
and by depending <mi die remote services from die Transact retrieval of further infonMtion relating to diat Action Data 
tion Data System 4g described below for die down-loading Element Upon receiving toe Y, Z information based on 
of additional software to die Brighteyes units and for die the stadium zero datum die hand hdd unit would convert die 
storage of information on betelf of ^)«afic Brighteyes units 50 data to its own za:o datum. The unit would do tiiis by 
between games. It is possiMe ttiat a number of different subtracting its own X, Y, Z (Xu, Yu, Zu) from die X, Y, Z 
versions of the Brighteyes units could be operating at any (Xe, Ye, Zc) of each Action Data Element as transmitted by 
particular event Since each unit has a unique identification the Action Data Syston 46. Thus die calculations Xe-Xu= 
number and can make diis number available over toe digital Xn, Yc-Yu=Yn and Ze-Zi^Zn give a new set of coordinates 
radio to die Transaction Data Svstem- 48-it is entirely 53 (Xn, Yn, Zn) wito die position of toe hand hdd unit as toe 
possible for die Transaction Data System 48 to provide new zero datum for each Action Data Element Otoer 
customized seivices to a large variety of different Bri^teyes methods of translation of axes may be equally effective. The 
units of varying configuraticm. Since toe exact location of unit would also use fee same calculations to produce its own 
toe seat within toe stadium occiqried by toe user can be location specific version of toe Stadium Model and generate 
uniquely identified from toe user's ticket required fOT 60 new coordinates for any graphics requested from toe Trans- 
entrance to toe park and toe X. Y, and Z position coordinates action Data System 4g. 

of toe seat in relation to a reference datum (toe "Stadium The location of the Brighteyes unit in terms of X, Y, Z 

Datum") can be transmitted to toe Biig;hteyes unit by toe (Xu, Yu, Zu) in relation to toe Stadium Datum such toat toe 

Transaction Data System 48 toe prefOTed embodiment of a position of die Brighteyes unit may be accessed by toe 
Brighteyes unit especially configured for this application 65 software within the Brig^yes computer. The pointing atti- 

does not require a position sensing system. An alternative tudc of the Brighteyes unit stored such toat toe heading and 

means for positioning toe individual Brighteyes unit using attitude of toe Brighteyes unit may be accessed by software 
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wiAiD the Bri^teyes ooiiq)uter. The focus and zoom posi- 
don of the lens is similariy stcH^ed so that both the focus of 
ttie lens and the zoom position of the lens may be accessed 
by the software within the Brighteyes unit Using the 
heading, attitude and location of the unit and the focal length 
and zoom positiOT of die lens the field of view would be 
calculated far the unit at that instant and the scene being 
addressed is then better known to the con:q>utcr. 

The X Y, and Z position of an Action Data Element in 
relation to the unit zero datum of the designation point on the 
boresight of the Brighteyes unit could be calculated by using 
the position and orientation data of the unit to project the 
boresight axis of the unit forward while searching that part 
of digital model of the park, Action Data Elements and 
objects within the field of view as calculated above to find 
&e point where the bcvesight axis intersects with the object, 
surface or Action Data Element. The user may put the cross 
hairs of the unit on a particular Action Data EXcmcst, surface 
or other image object and by pressing a "designate** key may 
select the item so selected for further processing. Multq>le 
items may be selected and variables relating to such de^g- 
nation such as location and time of designation recorded 
within the Brighteyes unit for further processing. To calcu- 
late the coordinates of the designated point in terms of the 
stadium zero datum the unit would add its own Xii, Yu. Zu 
to the coordiDates of the point thus reversing the process. 

ACnON DATA SYSTEM 46 

The purpose of tfie Action Data System 46 is to track and 
identify the location of each Action Data Element and to 
broadcast that information in real-time by wireless means to 
tttc Brighteyes units. 

In baseball examples of the Action Data Hements are; the 
b ^in play, eac h play er on the field (nine defense and a 
i Mumnm of four oirense), two base coaches, four umpires^ 
a^uTrmp, T OffP^g^ OT pitching coadL Typically tiiere age 21 
Ac tion Data Elements in basetiS T 

fhe preferred embodiment for the Action Data System 46 
is an industry standard personal con:q)uter with: at least 66 
Mhz clock rate; two "fi-ame grabbers**; a two way digital 
radio; a keyboard; a high resolution color screen equii^)ed 
with a light-pen or other pointing _svstem to enable an 
operat or to ra ^fljjLS ^ect items on the screen. Connected to 
the two franoe grabbers would t>e tw o hi^ speed color vide o 
ca ineras . Optional disks and peripherals could also be 
attached providing the primary mission of the Action Data 
System 46 was not reduced by bandwidth contention. 

The two'^tej ^ ^eea^^Sikisg^GafflefS could be located in -»i 
fixed and calibrated positions within a jstediom to enable tiic ^ 
position oT^^^^^^I^^^^^^I^l^^^^Eac^ 
^sgs^iSIESSSS^^^^i^si^^^^^g^^^ 
Sysien^Af. Eadi firame grabber 
would enable the computer processor in die Action Data 
System 46 to use stop frame action to track the location 55 
eadi Action Data Element in view. Additionally^ the Action 
Data System 46 could^S£ ery^c |a ff^ifrpmit h eiballmspe ed 
radar typically located immediately befaiiid home plate. - 
0->^ As each Action Data Element entered the field of play it T 
^ would be identified to the software of the Action Data eo 
System 46 either by che cking off a p re-arranged list or roste r 
or by the operator'^dcsignating a playcrT umpire, manager, 
coadi or unqiirc using the light-pen and indicating by 
keystrokes a unique identifier. The ball in play would also be 
designated in this way. Using the uniqu^identifier the Action 65 
Data System 46 would have access to'^ta relating to each 
player to allow the image processing software in the Action 



Dat a System 46 to t)etter analyze the st o p-frame images o f ( 
the "fflme and^ where required , to jenable the operator t oj 
resolve^ diffidigieOn^mai^ ^ reoogrution of play^ucfi" 
inf^magon"would ind udc th e dol^ of the uniforms, jhe 
heigb^f-flrgpaay ^lflft cr njjfffianaedffi jjay ani^ otha- 



infaffiSoSTrVpically the operator would be required to 
dcslgnatcThe general area where the ball would first appear 
firom the pitcher's hand so that die image processing scrfl- 
ware could commence t raddng the ball in play r^ teLthan 
those in use in the bullous or other visually similar and 
confusing objeas such as sea birds or garbage on the field. 



The op^tor may also be requirea to re-ocsignate players 
where two or more players crossed and/or doubled back at 
a plate or elsewhere on the field. 

Abemative technologies exist for various versions of die 
tracking function of the Action Data System 46 including the 
use of: precision radar, laser scanners, radio determination 
using passive or active transponders, and other more exotic 
techniques using interferometry. It may well be that a 
combination of techniques could be used to determine the 
position and identiiicatLon of each Action Data Element and 
that a more automatic and possibly more effective method 
may be introduced. 

Having determined the position and identification for each 
Action Data Etement the Action Data System 46 transmits 
this information over the digital radio to die Brighteyes units 
and to the T^sacdon Data System 48. . 
The Action Data System 46 would also transmit results of -1 
^ the previous play as it relates jp_statistics. Eadi piece of 
information relating to a specific pla yer would be tagged 
with a personal ID number: TBE girgga^ riSSiiMfci^^ 
b ^E^BltSgKE^5SSiS>nWj)a ta^^ga^ 
il^ ^it5rd a ^l§ato)indgdeitfaefDeTOi^^ and the 
Brighteyes hand held urdts themselves if the unit had, at that 
time, any of the different sgltsaacaiiiistsiofttheit iia^^ in 
question displayed. Software in the Brighteyes hand held 
unit would use this new data to update the displayed 
statistics, dius saving the bandwiddi necessary to transmit 
the players entire statistics after each play. 
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•^^^^^^^-D^^SYSl^f^ 

The purpose of the Transaction Data System 48 is to 
handle aU of the non-real time processing required to 
suppoit die remote Brighteyes units. Unlike the Action Data 
System 46 the Transaction Data System 48 is eqiupped to 
respond to individual requests from the Brighteyes units. It 
is also equipped to broadcast information, typically between 
ionings, to all Brighteyes Units. F^nthermorc, the Transac- 
tion Data System 48 is eqii^>ped with long-haul data Hnks to 
access systems in other stadiums and in other facilities. 

A version of the Transaction Data System 48 is an 
industry standard personal oonqniter in a configuration simi- 
lar to the -ADS but with the foUowir^ exceptions and 
additions. The Transaction Data System 48 does not have 
frame grabbers and associated video cameras. ToistCHSittieft 
^^lEESt^pq^' " * *"**'"^'"''™*^ ""''^^ fhf^ TYansaction Data 
System 48 is equipped with large memcMy. In addition to 
provision for one or more radios the Transaction Data 
System 48 is equ^ped with data links giving access to; other 
stadiums; ticketing systems; credit card centers; wire line 
news systems; gaming services; transpoxtation information 
and dispatch systems; and potentially other remote data 
services. 

The Transa ction Data System 48 tfxas^atiGcess»to*an 
e:a endet^ ei!Si6i;n^<>l^tl5?St5hita^W^ Stadium Model 
is a specialized data-base'IM mcEdes infomiation relating 
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to the l ocation in terms of X, Y, and Z of ever y a gnificam 
no^gliK5^ mg~o6iect md sigfa ce within fo e_gt adiiim . The 
^ the version that can be transmit- 
ted anfstored in each Brighteyes miit The extended version 
of the Stadium model includes the position of every seat in 
the Stadium and the location of every: concession; fire 
hydrant; electrical junction box; and other information relat- 
ing to the operation and management of the Stadium. From 
this extended version of the Stadium Model it is possible for 
the Transaction Data System 48 to provide customized 

of Brighteyes units (for example for 'foe-fighters, security 
services, and stadium maintenance) or by inean sjg42anted 
or o&CTjcgKffte^ Additionally the e^ndedj y,^5on of ithe 
Stadium Mgd^l-conld bem adc available such tfaat.ticket ing 
outletsTOuldJbeeqni pped^^^E^^^^^ fflS a^ 
ot faefif6c?!e^isrtoife!toiji^iegg^^ 
version of the view from the seat bdng sold or offered at 
point of sale. The extended and standard version of tbc 
Stadium Model co uld be created a nd updated fro m existin g 20 
lAa^ of tfacapi ^pnale'^aflHtog 'By using a fSngKteycs unit 
and the designate c^abOity outlined above it would be 
possible foT^^^!^^^^g^cjE£tSB!^^s^^Sy^ss^ 
ff StadtHni|Medelg^ /atfa«additionflb!a& 

pg^f^femanbn. 
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5 eTraiis action.Data.Syste]i3L4S would have a data-base- 
coDtaining i ^^DatiQ i^g^g fot^aU:tas p^ts^o 
tis&baJir:Maj^(lSlg^basebaH 
cc»c|irehensive^data-base-of Jacts.r^atingrtora^ 
the,g^e7.amd3his:infQ^ by the ^^^^^ta 

SystOT^^^-aftcr^eadi-play, would be avabablc via the 
Transaction Data System 48 to Bdghtcyes units either on a 
broadcast or individual query basis. 

^ A Bri^teyes uni t within range of a sta^iDoadiQi^^^ii 'J 
afi^^ ^^jgu^w onlgj^^^aiJpquji ?^^ 3$ 
^^S^^j^^tBeicmxc^tt^n^ ^ 
que^S^^noliQ^In addition to i ^onnation relatin g to cfaann^ ( 



position of the scat in relation to the Stadium Datum. This 
infonnation would then be transmitted to the individual 
Brighteyes unit for storage within the Brighteyes unit and in 
the case of a mislaid or stolen Brighteyes unit the informa- 
tion would be transmitted to Stadium security. There are 
other ways in which an individual Brighteyes unit could 
estabHsb accurate positioning with the Stadium. It would be 
possible to use tiie digital radio system to locate a Brighteyes 
transmitter by means of radio detenmnadon. Given the 
multi-path problem associated with a large enclosed area 
with many radio reflective surfiaces this determination of 
location would be non-tdvial and would potentially either 
lack in sufficient accuracy or may take an undesirable length 
of time. However, for a stationaiy Brighteyes unit it would 
be quite possible to locate &c unit with sufficient accuracy 
that fine tuning of the position could be accomplished by the 
user entering just part of the ticket number (e.g. instead of 
URS28, Row: 15; Seat 20 sin:q)ly use Row: 15; Scat 20) or 
by means of triangulation where the Brigjiteyes user would 
"designate" two points known to the Transaction Data 
System 48 which could then calculate the position of the 
Brighteyes Unit and transmit that back to the unit Alterna- 
tively the individual Brighteyes units could calculate posi- 
tion from similar triangulation to two points in the Stadium 
Model. Exploiting triangulation in this latter fashion may 
indeed be diosen as the preferred embodiment Additionally, 
a GPS equipped Brighteyes unit could operate within tiie 
Stadium, even in a covered stadium, assuming pseudo- 
satellites could be positioned within die Stadium or differ- 
30 ential &S was available in a suitable form. 

The T^ransacdon Data System 48 maintains a data-base of ^ 



Action Data from previous plays and can broadcastfceplays 
eitiier recorded from the ADS at the stadium or received by 



al locations^ friequen deS- and ig g|ggQls.to-be-Used.the,jajdip 
sc hedulc would d eflne-flie times and on which channel the : 
Stadium Model; software and software updates; inirialiring 
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Statistics; and other information would be broadcast In 
parallel widi this transaction the Transaction Data System 48 
would refer to the data base of Brighteyes units and would 
estaUish ownersh^ identification either from the data-base 
o r ^ggiiie ^iggatetBcfettt^ssBonltgtBa^^ 
flqniti-rtn'n ihYi^nnti'rjifiiffl^^ the Transaction Data System 48 
could not verify that the Brighteyes unit was in the hands of 
a. registered and a ntfaenticated user, then it would alert 
stadium security but otherwise proceed as normal 

Following user and Brighteyes unit identification and 
authentication, the Tir ansaction Data System 48 would query 
t faejggjeting system for inf oarmaticm relatin g to that user anJ 
w ould cstablislu as in the case of a season ticket holder, if a - 
seat number had been issued and was known to the system. 
The Transaction Data System 48 would verify with the 55 
Brighteyes user that the issued ticket was the seat number to 
be actually used or would request from the user for the seat 
number to be used. To facilitate correct input of the seat 
number (e.g.: URES28 row: 15; Scat: 20) the ticket could be 
encoded with a bar-code or other graphic and by allowing 
the unit to "see" the ticket the inf oonnatioh could be acquired 
from the ticket when held in the ^propriate position with - 
relation to the input lens of the Brighteyes unit Alternatively i 



commum gtion link from the Transaction Data System 48l t 
other stadiums. In tiie case of rq^ys from other Stadiums 
or^iecr^fiCg^so gg^affleventsifrtTO'tiieipas t the lYansaction 
Data System 48 wotdd preface the replay by broadcasting 
the stadium Model for that remote Stadium on on the 
assumption that there had been physical changes to tiic 
40 Stadium in tiie interim, for that historical play. These replays 
could be eitfa^ in the form of stick-figures or could include 
computer ge ngGat^ia ^^i«™tion"o^ tha ijprgr s in a^ on. The 
Transaction Data System 48 maintains a data-base of ani- 
mated infonnation. Computer animation of replays and 
otherneventsiQOuld be provided either by broadcast using the 
digital spread q)ectrum radio and/oar at some increased 
bandwidth to suitably equipped Brighteyes units. ^ 

INFORMAnON DBFLAYED IN PERSPECTIVE 
Having the above described hardware in place and 
operable, the system can now address a scene, collect an 
electronic image of that scene 52, identify the scene, gen- 
erate information relating to the scene 51, combine the 
con^fx^ generated infonnation with die image to form a 
coiiq>o6ite image 53. It is particulariy inqxirtant to consider 
the oonq>uter generated information. It is known to display 
*1ow battery^ indicia in prior art systems but the system 
the invention considers infommtion which is particular to 
the scene being addressed being identifiable to the counter 
by determination of the position and attitude of the device. 

It is further i]zq>artant to note that "low battery*' indicia do 
not have associated therewith any particular perspective. In 
contrast, the information of the invention typically is spa- 
tially dq)endent and therefore has associated thonewith a 



60 



the ticket numba- could be entered by the user using keys on t 

the Brighteyes unit Once the seat number is known to the 'dS particular perspective. FIGS. 6, 7 and 8 show a geometric 
Transaction Data System 48 a look-up of the inf ormation in 
an extended version of the Stadium Model will provide the 



shq>e in three different perspectives. It tiien can be appre- 
ciated from FIGS. 9. 10, and 11 that each seat of a baseball 
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stadium represents a diffaent perspective of the field. Com- 
posite images generated by systems of the invention arc 
presented in a perspective associated with a paiticular user 
and are aligned to the real scene such that the normal 
direction of the composite image plane is in the pointing 
direction of the camera. 

Special infcHmation types and features include: historic; 
statisticaL sym bolic^ a bstract symbolic, action tracks, histo - 
gram 

nf acfi ^n tracfcs. cQn^Hito' graphical interface devices, 
cTtScra. Jo 

HISTORY 



15 



20 



With systems of the invention it is possible to combine 
images from other time pedods onto live images of real 
scenes. Historic recordiii es can be played-back with live 
scenes. Also, oonq)uter generated **futuristic*' scenes can be 
combined onto real images. 

If a user of the vision system desiies to see how it 
appeared when Hank Aaron broKe the career honoe runs 
record in Ftilton County Stadium in 1974, the images from 
that time can be recalled fipom tiie data store, adjusted for 
proper perspective according to the users position and 
attitude, and replayed onto the field as the field appears 
today. A rq)lay of a film sin^y shows the perspective as it 
was recorded on the day of the actual event. B ecause that 
pc^pective may have no relation with die viewers present 
perspective, the replayed film is like sinq>le video play-badL 
To give a strong feeling of 'tjeing there** systems of the 
invention aHgn the events of a historic even t to &e field as 
it surrounds ^e user. If the user chooses to watch the ball fly 
over the fence, then the camera is pointed into left field as 
the ball jQy'5 over. If the user decides instead to watdi the 
reaction of Hank Aaron^ the device is pcanted to the home 
plate — first base line. In cither case, the viewer "sees* the hs 
historic event superimposed with the present stadium. 

In similar fashion, events that have not yet happened but 
that are generated artificially c:an also be **repiayed" onto 
present scenes. This can be used to 'imagine" **what-if 
scenarios. ^ 



other statistical feature. If one moves firom viewing the 
triangle from directly "above" or normal to the plane of the 
papCT, then the perspective of the triangle changes FIG. 7. 
FIG. 8 shows how the same triangle looks at a glancing 
angle off to the right side of the apex. The shaded area, 
perhaps representing a region associated with some statis- 
tical information changes as well. In all three perspectives, 
tiie shaded area indicates a particular area of interest It is the 
same area in all three figi^. 

This can be further illustrated by considering what a 
baseball field looks like frcm each seat in a stadium. In 
baseball it is common to present statistical data associated 
with a portion of the field. IF a particular batter frequently 
hits to left field, a display can be arranged to indicate the 
trend. 

Users of the system are distributed throughout the stadium 
seats and each has a different perspective of die field. By 
looking into the display of their Brighteyes sysitem with tiie 
device pointed in the direction of interest, the Action Date 
System could instruct the units to shade the left field as 
defined by a particular set of coordinates. When the coor- 
dinates are plotted, the resultant area will ^>pcar differently 
for every Brighteyes unit 
L.L As a new batter that is up to bat is identified, thec^gutgr 
c ould find that the batter has hit the current pitcher to le ft 
25 fi eld ^0% of fee time . The Action Data System 46 then 
transmits an instruction to shade t he left field by defining the^ 
region with three points. The 



stai^^ ^mfprmgaoQggndiQiej 
^ w fflBgSMt o lB^shadeyl g l^^pum 

Pi gjitg@^^TO&gt>ec^^£th^^ffieldr will uniquely represent 




REFRESENTAnON OF STATISTICAL DAIA 



Statistical data can be represented in many forms. "S0%'1 
of all 0300 hitters are heavier than 220 pounds" is a simple 
numeric (80) representation of infomiation regarding a 
player's weight Sometimes information can be presented in 
a gr^hlcal form which relates the data to the subject of the 
data. For exairple an outline of a player's figure that is 80% 
shaded on the inside. Presentatioo of data in relation to 
images may give the user a better feeling of its significance 
and may be easier to interpret. 

Wh^^tisjd^Ldata relates.to.a.g^gr^}hic.ar!E^ t he date * r 
may be presented su perimp osed onto flie geogra phic area. as 
on eViews it.fram various perspectives^ By using the system 
of the invention, one looks into a display having aurimsge 
otaTrcalTSceneTalig nedio ^gi&.pQsition and attitude of the user 
w^^respect,tOpt hat^ scc ne^The perspective of the scene as 
viewed normally looks the same as the perspective of the 
image displayed when the system is ptnnted towards the 
scene. An area of interest within the scene may have 
statistical data associated with it In this case, the display can 
supoimposc indicia onto or around that area in a predeter- 
mined way to rq^scnt that data. 



the left field on its display in accordance with the perspective 

as Viewed from the user's loadion. Alfliough die left field is 

a simple two dimensional area, statistical information can 
also be represented in many three dimensional forms. 

This can be further illustrated by considering a players'l 
defensive range in a baseball scenario. If we choose a 
probability threshold of a player responding to a ball in play 
as a function of position near the player, and indicate where 
that threshold lies, the limit would surround the player in the 
sh£^ of some dome like fccm (FIG. 12). Extending perhaps 
further in the direction ahead of the player than in the 
direction behind (this is due to the fact that ag tecotgan ^ qiPL 
ri^gE^^ifytlglldcwardrdircaigaiffi^ 
^^SSSS). Whereas a player wears a glove on one hand, 
there may t>e some bias to either side, (backhand plays are 
more difficult than open handed* plays.) This coukl be 
predicted by a history of plays made for a certain player or 
by averages for a certain league or team. The dome, being 
50 determined by whstevex means, would be embodied by a 
some surface wifii the baseball player inside and the baseball 
field completing an enclosure. This enclosure would appear 
diffiBxently from every seat in a stadium, the drawing in 
FIG. 12 existed as a printed hnage ifi a game program wiA 
an explanation of a centeifielder's range, it would be 
eactrcmely difficult for a fan sitting in the outfield bleachers 
to translate where on the field in the direction of second base 
corresponds to &e limit of the fielders range. However, if the 
data of FIG. 12 were plotted with the propor perspective 
^ according to a particular fan viewing the drawing and 
aligned and superimposed onto an image of the real scene, 
then it would be trivia] to interpret the field locations in all 
directions which define the players defensive range. 



55 



Considering FIG. 6, we can imagine looking down on a 65 SYMBOLS 
right triangle defined by the perpendicular black lines. The A common problem shared by most attendees of stadium 
shaded area could indicate some high probability density or Events is the difEculty of locating facilities sudi as the 
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lestrooms, concessions, or police. Because a laige stadium 
event could be visited by more than 100,000 pcopk, it is 
easy for an individual to become lost Particulariy annoying 
is the problem of locating one's car after an event. As the 
stadium paiMng lots are generally very large and contain 
many similar looking cars^ it is quite diffioilt for one to 
relocate Iheii parking place. Since users of the systems of the 
invention will have a device enable of locating itself and 
having previousl y recorded fce ^position of th e^parking 
space, by pointing ana scanmn govar large areas of a parjd ng 
lo^^ey wEITBeable to look into the display to see an image 
of the parking lot with an Indication of their parking space 
hig^ilighted. Similarly, the other fadUties of the stadium can 
be located by lookiog into the Biighteyes unit Symbols can 
be used to indicate locations Hke a rcstroom hidden by a wall 
or image features that may be too small to be seen. 

ABSTRACT SYMBOLS 

JO Foul ball planes are abstract in the sense they exist as an 
invisiMe plane but do not rq^iesent any physical object An 
umpire and other observers can judge with some degree of 
uncertainty when a €a i]^aS5 earf biaatth , ^a«fo>tl »hati«ntaTie. 
However, since it is a trivial task to plot a plane onto an 
image in systems of the invention, the uncertainty of foul 
ball calls can be completely removed. Of course, each user 
of the vision system would sec the jJane in the correct 
perspective according to the location of the user. It is^ 



to 
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CT ^ridemisignalri^tWcondit^^ 
the path of a foul ball W hile watcfai ^_a ball hit toward t he 
foul ball area of Jhe-field, a^d^HK^ gggp]^ ^ 
rt>p han~5nTrHi^ftlY hp. a foul ball and indicat e an e^Iyalann. 



I and mi 

The foul ball plane 143, left invisible~during normal play, 
mniH jii^tnTTiariraiiy be^Jottcd outoan im^c ofthe field in 
a 'SvarnS^ . ^^ Kteydlow. As t^T^all passes'Quough the 
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another. One pitdicr may have curve balls that break dif- 
ferently that others. Ihese features could be readily illus- 
trated in devices of the invention. 

IMPROPER PERSPECTIVE 

It may be difficult for some viewers of a curveball pitch 
to apprcdaxe die difGkiilty of hitting the baU. Prom nearly 
every view in the stadium, die ball ^Tpears to have a 
trajectory leading into tfie strike zone. However, the catcher 
knows differently. He (and the batter) can very distincdy 
notice a one foot breaking action. 

Since a balls location can be known and mapped with 
precision during the course of a pitch, the path can be plotted 
onto the coordinate system from any perspective selectable 
by the useL Of course, a preferred perspective for viewing 
a curveball pitch is likely the catcher's, any of an infinite 
other perspectives can be selected to view the pitdi. 

If a curveball breaks strongly left-to-right but not at all 
up-and-down, dien the path of the ball could ^pear to an 
observer In the stands to be a straight line (actually parabolic 
concave down). If one chooses to view the pitdi from 
directly above the field, a position relatively difficult to 
realize in an actual game, (although the Groodyear blin^ 
may argue with this) by selecting that position one can see 
how much the pitch breaks left-to-right These are two 
illustrations where the users perspective is not preferred, a 
b^ter perspective is known, and the user specifies a 'Virtual 
position** for displayed images to be translated. 

PANELS 

Special "windows*'-type con^xiter graphical interface 
objects herdn called "Panel"s are used in images for various 
purposes indnding information selection, system command 
and control A Panel can consist of a list of related 
infOTmation. selections and/or options. Panels are image 
objects that can be related or contain information to some 
spatial feature of the game of basebalL A *fixed* Panel is a 
Panel tiiat is s tatic relative to the background or^ .designatRd 



plane FIG. 14B, the coIct: of th^pjaB g^could cfaange to red 
144 toj ndicate a foul balL ^on^d^^c^f^y couId"£ar&- 

f oredrive the features being displayed on a Brlghteyes unit agti on elemen TSome Panels may be autoioatically fixed 
Sy mbols to indi^ te abstract objects like J <y3LbaU,planes t the boiesi^t of die unit was pointing when the Panel 

couldbeplottedontorealimagestoenhancctheinfCTmatian j was accessed, and other Panel may be placed and fixed by 
presMteito£usCT. ^ ^ If ^ p^j^ ^ by die user the Panel would 

^]pear as a semi-opaque, flashing JmajBC- on the boresi ^t 
and would mnain on the borcsight until the user had 
positioned the Panel at the desired location and pushed the 
designate key on the unit The user may fix a panel on a 
specific heading, on a q>ecific heading at a specific range or 
at the locationof ph ysical ob j ect tfa atis- jnovuig , if the unit 



BALL TRACKS 

A curveball pitch can be briUiantly executed by some 
pitchers, sometin^s breaking more that twelve indies. Since 
a baseball is only in one point at one time, it is quite difficult 
to envision the exact path of a pitched basebalL If the 
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pos ition of a ball is reaxmled^as the ball approaches th e r eceavesjooSon data f ot that object from the 



batter, a plot could be presented to the display whidi shows 
the padi of a pitched balL Hie Tnaximum lateral displace- 
ment could easily be computed to determine the breaking 
quality of the pitch. 

HICTOGRAM 

As each pitch is different, a I jistory j>f scvcr d ot many 
pitches by one pitcher could be dispiayea snmiltaneously 
yielding a histogram of pitches. Because the int^^sting 
property of a fast ball is speed compared to position, the 
histogram could be arranged to exclude those type of 
pitches. Displaying the paths of tfic last three curvcballs 
pitched would yield a useful histogram. FIG. 15 shows a 
pitcher 151, on the mound 152, the home plate 153 and the 
batter 154. The last three curvebaU 155 pitches can simul- 
taneously displayed as a histogram. It may also be instruc- 
tive to compare the average curveballs of one pitcher verses 
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System 46, or stationary. The Panel would stop ff^^ng and 
remain in this fixed position until moved or dosed. To 
designate a selecdon in a fixed Panel the user would move 
the unit as a vWiole until the desired selection was located 
under tiie crosshair and then push the designate key on the 
unit 

Panels may t>e displayed in the form of a two sided '*sign** 
with Imdles** to rotate it atxmt its central hocrizonta] axis. 
The left "handle" would rotate the "sign" 180 degrees to 
60 diq)lay the other side, the right one 90 degrees to present the 
edge of the "sign" to the user so as to tempOTarily clear the 
area f ot "live" images. 

When the Panel was edge-on to the user it would look 
akin to a barbell witii the "handles" connected by a line and 
two tide bars, one above and one below, floating next to it 
The left handle would contain an up arrow, the right a down 
arrow. By designating either handle and pressing the desig- 
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nate key tiie user may rotate the Panel to display either side. 
To move a Panel, in either case, the user would designate 
either title bar and press the designate key thus tying the 
Panel to the crosshair, move the Panel to the new location 
and press the designate key again thus fixing the Panel in the 
new locaticMi. To close the Panel, Le. to remove the Pand 
from view, the user would place the crosshair over the 
"Remove** button in the Panel and push the designate key. 
The unit would fix the Remove Panel in front of the user 
consisting of the questions, **Are you sure?**, and the answer 
options: "Yes", and *T^o**. If •'Yes** is selected the Panel in 
question would be removed fi^om view and the **Rcmove 
Panel** would be deleted. If **No" is selected the unit would 
delete the Remove Pand. Hie unit treats the entire surround^ 
ings as a "screen** and the user looks at different parts of the 
"screen** to locate the infonnation they desire. Infonnation 
could be fixed in the sky, on the back of someone's head, on 
billboards,, any position the user could see with their Bright- 
eyes. Panel may be "stacked** over one another, lb move a 
Panel to the ^'top*' of a "stack" the user would designate 
cither title bar, as if to move the Panel, and press the 
designate key again. 

Panels may be "attached** and follow the dynamic Action 
Data Elements. They may be used to display images as well 
as menu data. FIG. 13 shows a '*replay- Panel 131 fixed in 
the outfield portion of an image of a real scene. The Panel 
is displaying the play which happened moments ago onto a 
"live" image. 
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System 4S would tag the unit numbo- and when the unit 
reported a seat location the Transaction Data System 48 
would notify security 222 as to the location of the suspected 
stolen unit 

If the unit was not reported as stolen the Transaction Data 
System 48 would then recall 223 the user ID number for the 
unit K the ID number that was inputted by the user did not 
match 224 the stoied nQmt>er the Transaction Data System 
48 would request a new input 225 from the Hand Held Unit 
twice. After the third attempt die unit would go into Prc- 
Game Mode, bypassing the credit check, but would report to 
security 226 that an unauthorized user had chedoed into the 
systent The Transaction Data System 48 would tag the unit 
number and when the unit reported a seat location the 
Transaction Data System 48 would notify security as to the 
location of die suspected usee 

The Transaction Data System 48 would then check 227 
the users credit to ensure there were suffident funds to pay 
for the air time of the game. If suffident funds were not 
available the Transaction Data System 48 would ask the unit 
to request 228 another credit card number firom the user. If 
none were available the unit would be told to shutdown. 

If the hardware ID nnmber and the personal ID nuinbers 
matched and there was suffident credit available the Trans- 
action Data System 48 would check 229 to see if it had any 
record of a ticket f that user for that specific game. If there 
was a ticket number the Transaction Data System 48 would 
transmit the X, Y, Z coordinates of ttie seat to the Brighteyes 
hand held unit where if would be stored in the units intonal 



Three examples of composite images which are com^ 

prised of an image of a real scene and computer generated IdO memory and used to generate the units own personal set of 

augmentation are presented as FIGS. 17, 18 and 19. FIG. 17 1 coordinates in once the user was seated, 

— 1 ^ *i 1 — J 1 Once this sub-routine was completed the Transaction Data 

System 48 would download 213 all die neccssaiy software 
to run die Baseball Application to the unit and the unit would 
activate Pre-Game Mode 214. 



40 



45 



shows defensive players and the playing range of those 

players, FIG. 18 shows an image of coin)Utff generated hall^ 

tracks representing the last four hits of a particular batten 
superimposed onto an image of a baseball field. FIG. 19 
shows a oonq>uter generated '^strike zone** laid onto an 
image of a real scene. In addition, indicia rdating to the 
batter may be dispLaycd as text overlaid on an image. | 

Wc teach the system operation of a typical ^nbodimert 
with &e help of flow diagrams to guide tiie reader through 
relationships of the various systems. We present the flow 
diagrams in various levels^ the low numbered levds corre- 
sponding to fundamental processes and the higher numbered 
levels corresponding to detailed sub^irocesses. By correctly 
placing all higher level flow diagrams into their appropriate 
places in low level diagrams one could form a single flow 
diagram. The division onto many pages is for clarity. There 
is one continuous flow diagram. The flow diagram shows 
five modes of operatioiL The five modes are: initiate, pre- 
game, game, post game, and shut down. FIG. 16 shows how 
the levds cooperate. INmATE, FIG. 21 

When the Brighteyes unit is first turned on it would go 
through a scries of self tests 210 to check its hardware 
integrity and memory. It would then report unit health to the 
user aiKl retrieve the hardware identification number from. 55 
intemal non-volatile storage. The unit would then request 
that the user, dther orally or by way of a Panel, input their 
personal identification number 211, The unit would transmit 
bodi ID numbers to the Transaction Data System 48 via the 
two way radio link. 

The Transaction Data System 48 would check 212 to see 
if the unit had been reported as stolen 22L If the unit had 
been reported stolen the unit would notify security that a 
suspected stolen Brighteyes unit had diecked in to the 
system. The Transaction Data System 48 would bfypass the 
Personal ID number and credit checks and appear to go into 
Pte-Gamc Mode 214 as normal. The Transaction Data 
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PREhGAME mode, HG. 23 

Prc-Game Mode is the mode of operation the unit oper- 
ates in until the user changes to the primary mode of 
operatioiu Game Mode, and is activated automatically when 
the unit is turned on by the user. Pre-Game Mode has six 
Main Menu Options that allow ttie user to access informa- 
tion and/or interact with the unit 

SAVE ARRIVAL POINT 230 

The Save Arrival Point Main Menu Option allows the user 
to input the point to which th^ would like to return to after 
the game is over. If the point is the location of a car the user 
5Q would input the code for the parking space, i.e. Red, Row 34. 
Bay 23 by either designating flie parking space code or, if the 
unit cannot process flie image, manually inputting the code 
via a Panel If the point was a bus stc^, train station, w taxi 
stand the user would iiq>ut the Brighteyes location code of 
tiiat point by designating the location placard posted at the 
location or manually ii^>utting the location code via a Pand 
if the unit cannot process the image. If the unit was equipped 
with a GPS recover and the point was outdoors the location 
could be saved as a latitude-longitude position. The unit 
would store the location in its interiMl memory. 
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INPUT LOCAnON 231 

The Input Location Main Menu Option allows the user to 
input thdr location within the stadium before they get to 
thdr scat Locations around the outside and concourse of the 
stadium would have Brighteyes location placards, consisting 
of a number code, posted. The user would activate the Main 
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Meau Option and designate 2S0 the placard. The unit would 
process 251 the image, read the code, transmit the code 252 
to the Ttansaction Data System 48 and would receive 253, 
from the same system, an X, Y, Z location in relation to the 
stadium datum. If the unit cannot process tiie image* the user 
would input 254 the location code manually via a Pand. if 
the unit was equipped with a GI^ receiver and couM see the 
sky the location update could be continuous up to the point 
that the unit loses GFS positioning. If the unit was equipped 
with an audio in/out system the user could activate the Main 
M^u Option and read the location placard data out loud 
enabling speedb processing software known to the state of 
the art input the location code. This may t>e in addition to or 
in substitution foi otiier means for dpfining location within 
the preferred embodiment The unit then stcres the acquired 
location data 255. The program flow then returns to the pre 
game mode main menu 256. 

INPUT SEAT NUMBER 232 

The Input Seat Number Main Menu Option allows the 
user to input a new seat number, if the user is a season ticket 
holder and not sitting in dieir usual seat, or input their seat 
number for that game. The user would activate the Main 
Menu Qptioa, hold the ticket up and designate 260 the ticket 
The unit would process the image 261. read the scat number, 
transmit the number to the IVansaction Data Systeni 48 and 
receive 262 an X. Y, Z coordinate for that seat ntunber in 
relation to the stadium datum. IF the unit could not process 
the ticket image, the user would manually input 263 the seat 
number via a Panel If the unit was equ^iped with an audio 
in/out system the user could activate the Main Menu Option 
and read the seat number data out loud enabluig speech 
processing software to iaput the location code. This may be 
in addition to or in substitution for other means for defining 
location within the best versions of the system. 

DIRECnONS233 

The Directions Main Menu Qi^oa allows &e user to g^ 
directions to their seat &om the last saved location. When the 
Main Menu Option is activated the unit would check to see 
if a seat number has been saved 270. IF one has not then the 
unit would prompt 271 die user to input the seat number The 
unit would then ask 272 if the location has changed. B'ithas^ 
die unit would pronqjt the uso- to input 273 the new location. 
The unit would then ask die user to fix the IHrections Pland 
on a particular bearing so that as the user moves the Panel 
would move with them. The Panel would contain written 
directions 274 to the inputted seat number. 

STADIUM INFORMAnON 234 

The Stadium InfotmatioQ Main Menu Option allows the 
user to have icons rqxresenting various services available at 
the stadium fixed over their position. When the Main Menu 
Option was activated the unit would, if not getting location 
data from GPS, ask if the location has changed. If it has. the 
unit would pronq>t the user to input the new location as 
described. The unit would then fix icons representing the 
closest men*s and ladies t(Mlets. ticket office, security office, 
etc. at their locations in relation to the users position. 

GAME MODE, FIG. 29 

Game Mode is the primary mxxle of operation for the 
baseball application. It is active from the time the user 
activates it tiU the time that the user designates the Post- 
Game button on the Post-Game panel that the user fixed after 
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the last out of the game. There afe ei^t Main Menu Options 
in Game Mode, each allowing the user to interact with the 
unit and recall a myriad of information should they so desire. 
The Game Mode consists of die following routines: setup 
5 290, anto, coach, scout 291, offense 292, defense 293, 
pitching 294, teams 295, stadium 296. and FUN! 297. 

SETUP 290 

The Setup Main Menu Option allows the user to select 

10 and customize the way thst basic information about the 
game is presented. Once seated, the user would go through 
the ^setup" procedure for that specific game. The unit would, 
display and fix 310 the Setup Panel in front of the user. This 
Panel would define how basic information relating to the 

15 game would be presented to the user. The Panel would 
contain a list cf options, some of them, if turned on, 
generating user fixable Panels of their own. For example, a 
user sitting along the third base line designates the fix 
sox-eline button on the Setup paneL The scoreline would 

20 appear in the center of the view. The user would then move 
the entire unit untO the scoreline was positioned on the right 
field walL press the designate key and then be able to look 
away, leaving the scoreline fixed on the waU. If "screen 
edge" option was designated, the infonnation would appear 

25 along the bottcxn of the view. In this way the user can totally 
customize how they view the bask information pertaining to 
the game. The user could place the scoreline on the fence, 
the lineups on the backstop, the names of players left on the 
bench over the dugout, the names cf players in the bullpen 

30 over the bullpen, the batters average and name over his head, 
the count, score, outs, and innings over home plate. This 
allows totally customized viewing wi&out the need for a 
scoreboard. All the information, averages, linei^)s. etc., 
would be downloaded by the Transaction Data System 48 to 

35 the unit throughout the game via the digital radio link. Hiis 
inf cmnation would be updated by the Action Data System 46 
and would change as the game progressed. The user may 
return to the Setup Main Menu Option at anytime to diange 
the parameters they have previously selected. The user may 

40 save their favorite setups for each stariinm they visit If the 
user has a setup saved the unit would ask if the user would 
like a previoudy stored Setiqx If the answer is yes then the 
setup automatically takes pdace. If the answer is no then the 
ulser would go through the setup procedure. It is possible that 

45 there could be several setup (^cms saved for each stadium 
from whidi the user could select The user may also enter a 
new seat number in the S&cap Main Menu Option if they 
moved during the game, or the unit was being used by a user 
in aiK>ther seat The user may also save the existing set-up 

so at any time by designating save Setup die Setup Panel. To 
»ter die new seat number the user would select the Setup 
Main Menu Option, designate the new seat number button 
and enter the new seat number. It is also possible tiiat there 
be a "Multiple User" <^)tion inthc set up Main Menu Option. 

55 If this option were selected by the user die unit would ask die 
user *1]ow many users today?". If the user entered four then 
each person who would be using the unit diroughout the 
game would be given a number one through four to identify 
themselves to the unit They would each go through all of the 

60 steps in the "Set Up** Main Menu Option and the unit would 
remember all of ^eir individual parameters. To switdi 
between users the user would locate the Mult^le Users 
Panel and designate the desired number button, or of the 
number of die user who wanted the unit next As soon as the 

65 button was designated die parameters change and the 
gr^hics, and Panels would be seen from the point of view 
of the new us^. The user may also select the Auto Option 
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380 wbidi would execute a default setup. The Auto option aim." The infannation would be stared either in the unit 

may be turned on and off at will during the game by entering itself or in the Tiransaction Data System 48 and would be 

the Setup Main Menu Option. conq>iled through interviews with and analysis of past 

games by baseball experts. 

^^^^ ^ AUTO OFnON 380 

The Coadi option in the Setup Main Menu Option gives The Auto Main Menu Option would allow the user to take 

the user access to options on drfensive and offensive align- advantage off all the diflFcrcnt aspects of the Brighteyes 

mcnts and informed predictions on flic play of the game. Basdjall Application without having to touch another key or 

When Coach is activated the user would fix the Coach Pand ^^j^^ ^^^^ ^ i^Qj^ through the unit 

in the desired location. The Panel would consist of a ^jpropriate momait, and at the appropriate place to 

selection for each team. When a team is designated Coach pre-programmed infonnation, in the form of text or 

depending on ^^1lethe^ that team is batting or fielding, would gr^hics, the unit would be dij5)laying foi the user. The unit 

explain the defensive alignment of the fielders or^e^a&e ^ould either access information it needed jfrom it's own 

prqbableoffensive strategy.otaifeJ6am. When the Coach is memory, (ff download it via the two way digital radio link, 

on he wiffalso interject during the game e.g. "Watch the guy Option would stiU aUow the user to set seat 

on first he* s a flyer.". The information could be displayed in location, so it could be changed throughout the game. AU tiie 

the left side of the view as a speech bubble coming from a j^^^ Options witii the exoq)tions of S^p and Auto 

caricature coach, or the report may be in the form of a full would be locked out until Auto had been deactivated. In the 

motion and sound film ctip. The speech bubble would stay ^ j^^^y Option the only active options would be 

on the screen for a finite period and then ^dppcsr. The relating to new or mult^>le seat numbers. If the unit 

information would be stored either in the unit itself OT in the ^ equipped with GPS and was receiving positional data 

stadium database and would be cwnpiled through interviews system the Setup Main Menu Option would also be 

with and an^sis of pastgam ^_by baseball expe rts. locked out until Auto was deactivated. To input a new or 

>Qi 25 additional seat number the user would select the ten^orary 

SCOUT 2^1 Merai Option. The same method of seat selection 

The scout option in tiie Setup Main Menu Option would described would be used. The "SeaT option would not be 

give the user access to ejqpert opinion on the player*s cr necessary if the unit was equipped with GPS, and was 

team's off^ensive or defensive abilities. When the user des- operating in a stadium that had GPS pseudo satellites. The 

ignates the Scout a Panel that defines whidi player and what 30 only option left in tiic main menu in this case would be "Set 

type of report would appear in fircm of the user The user Up". The unit waild display a set selection of Panel and 

would then fix the Panel and select the desired option by icons througjiout the entire game. The user would go 

momg the unit as a whole, phdng the aosshair ovcx the through the placement process to fix each Pand in the 

option and pressing the designate key. The '"Visual Destg.** desired location in the stadium. Tliis allows the user to 

option would be automatically active, until "Other" was 35 customize viewing for their individual point of view, 

designated, with the crosshair flashing. The user may then aU the information would be constantly updated and 

either select a player by placing the crosshair on that player; displayed automatically by the i^nlf The unit would receive 

if be is visible, or designating his name in one cf the player infonnation from its memory, and transmissions from the 

lists, lineup, bench, bullpen, that could have been placed stadiums Action andTransacdon Data System 4S4Ss needed 

around the park in the setup stage and then pushing the 40 for the display. When tht Auto Main Menu Option is 

designate key. If the user selects "other** a new Panel would deactivated the unit reverts to the set-up that was active at 

^jpear and fix itself in firont of the user. The Panel would the time Auto was activated, 

also enable the user to select a player from a team roster that /^mtrKrec w» 

has not already been fixed. OFFEND 2»2 

If Home Visiting team is selected the complete roster 45 The "Offense** Main Menu Option gives the user access, 

of that team would fix itself in front of the user, the user graphically and statistically, to information regarding 

would designate which player with tiie crosshair and the Offense pUy about the selected player or team. When the 

Panel would dose. If the user selects "other team" the k>gos user selects 400 Ae "Offense" Main Menu Option byusing 

(rfaU teams would fix themselves, ordered by division and the scrolling key and pressing tiie Main Menu QpUon 

recOTd, in front of die user; The user would tiiendeagDatB 50 activate key die user would asked to fix 401 the "Cffense 

the ^K>ropriate team and go through tiie steps previously Panel where desired. On one side tiiis Panel would allow the 

described. By choosing one of tiie options the user would user to define 402 the player or team tiiey are interested in, 

receive cfther an offensive or defensive scouting report on turn on cff off the statistics and gr^hics options and tiic 

die desfred playw or a short biography of tiie player. Tlic "Statistics Parametets" sub-PancL 

report would dtiicx^)pcar on die left side of tije screen and 55 On tiie otiier side of tiie ^tMfense" Pand the statistics as 

remain tiicre until tiie user closed ttierepwt or be in ttie form defined in tfie "Statistics Parameters'* sub-Panel would be 

of a fihn cUp of a coach, for exan5>le, witti full motion and di^layed along witti a Batting/Running button- Batting 

sound. D^nding upon the memory size of tiie unit, tiie statistics would consist of RBIs, runs, batting avg., hits, 

infcttmation for tiie scouting reports could be stored witiiin hcMnc runs, ct cetera Running statistics would consist of 

die unit or accessed from the Transaction Data System 48, 60 stolen bases, attempts, runs, caught stealing, etc. As different 

via tiie two-way radio link, as needed. A variation of this players where designated the Panel would change to display 

would be to store tiic reports on the two teams playing tiie the statistics of tiie designated player, 

game in tiic unit and only access tiie Transaction Data If "Area Graphics** 411 was turned on ttie unit would 

System 48 for information on other teams. An enunmplc of an display the area of tiie field that ttie ball is most likely to be 

offensive scouting report would be, "Dead pull hitter. can*t 65 hit into. This area would be viewed from tiie users perspcc- 

lay off the high outside fastball.** An ^lample of a defensive tive and its location would be defined by the parameters set 

report would be, '^Covers a lot of ground but has a weak in the "Offensive Graphics Main** Panel. 
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If 'TUH Gr^^Mcs** 412 was turned on the unit would show, distance from the plate and velocity of the batted ball This 
in the form of coicred lines defining the path of the ball from is illustrated in FIG. 10. Before the pitch is thrown, the 
the batters strike zone, aU the hits and outs, and where and defensive bubble would be displayed as defined in the 
how far they go, as defined by the parametCTS in die "Graphics Parametors" Panel If the velocity of the struck 
*t>flfensive C&aphics Main" Panel. The view of these Hncs 3 baU can be measured, and the dau sent to the unit, in near 
would be specific to the perspective of the usct. The user real time the dffensive bubble would shrink or grow accord- 
could sec the tracks of the hits/outs as they would appear ingly- When "Graphics'* toped on for flic &st time flic unit 
from his specific positioa would ask the user to fix "Graphics Parameters" Pand. This 
, . . . ' Panel would allow the user to select the players who have 

When eitha- of the Graphics options was turned on for the bi^bles displayed and also select the strength of the hit 

first time, the unit would ask the user to fix 413 die "Offense 10 This Panel, would define Ae configuration of the defensive 

Graphics Main** Panel The best location for this Panel bubbles before tfic ball was struck. The information would 

would probably be "stacked** with die "OfiFensc" Panel. This be stored either in the unit itself or in the stadium database 

Panel consists of all the different parameters that zScd the and would t>e compiled through interviews with and analysis 

offoisivc gr^hics di^layed. The sub-Panel in this Panel of past games by baseball expem. 

would cover parameters ranging fr<Mn the time period to ^5 PITCHING 294 
pitch location and include a default siting. When the user 

has nffide the desired selections, or dianges. the user would 'Titching" Main Menu Option gives the user access, 

select OK and the unit would display toe mpropriate graiA- ^ graphically and statisticaUy, to pitching information 

about the selected player or team When the user selects the 

' ^ • ^ 2D "Pitching" Main Menu Option the user would be asked to fix 

The balltracfcs or hnes would be stored m the unit OTin the pitching Panel On one side fcis Panel would allow the 

stadium database. They would be stored as digital informa- ygg. ^ <iefine the pitcher cr team they are interested in, turn 

tion and would be entered dtfaer by an ultra fine radar on or <^ the gr^hics and statistics options and set the 

tracking system in each stadium, or by one or several data "Statistics Parameters'* sub-Panel 

entry personnel at the stadium as die game progressed. On the other side of the Pitching Panel the statistics as 

Initially the tracks would have to be entered manually to get ^ drfincd in Ae Statistics Param^ers sub-Panel would be 

the database going. As soon as the database was operational displayed. Pitching statistics would consist of ERA, won- 

having all information, statistics, results, balltcacks for play- lost rec(sd, strike outs, walks, wild pitches, iimings pitched, 

ers still in the league, ball tracks for great players from the run against, saves, appearances, eta As different pitchers 

past, etc. for the entire history of baseball then the infor- where designated the Panel would change to di^lay the 

mation would be entered up to the second in each stadium ^ statistics of the designated pitdier. 

across the country or world. The databases in each stadium If "Area Gr^hics** was turned on the unit would display 

would be linked to each other and they would all receive the the area of the fi^ld that the ball is most likely to be hit into, 

same information. Itiis area would be viewed from the users perspective and its 

•n.eiiifonnationneededforthedispUyediteins.statistics, „ i^*"^"'!.'^ ^"L'V P'"^'^ set in the 

ban trajectcdes, etc.. would be eithe* stored in the memoty ^aphics Mam PaneL 

of the unit, or, in the case of a unit with linuted memoiy, ^ » G««P^ «° "JP' ^^1* ^fl' ^ 

downloaded via the two way digital radio link from die «f ^ If^l?^ pi^ erf the ball from 

_ . , . ' jCII . ^ the batters strike zone, all the mts and outs against the 

^^^^^^J^'^'^^f^^^ f selected pitchy, and ^ere and how far they go. Is defined 

n.^^^"^ imng mformauon downloaded via the ^ ^ p^ametm in the "Pitching Graphics Main" Panel 

digital radio It could ddrte it from it*s mmory and if ^ ^e 5)ecifiVto the perspective 

deleted the omt would have to access die database agam If of the usct. Ic. The user could see the tracks of the hits/outs 

die same infwmation was needed tiie user. ^ ^^^^ ^ ^^^^ position. 

DEFENSE 29^ When cither of the "Gr^hics** is turned on for the first 

45 time the unit would ask the user to fix the *Titcfaing Graphics 

The "Defense** Main Menu Option gives the user access, ^j^y* ^^^^ probably be "stacked" with 

botfi statistically and graphically, to defensive infonnation die *Titching" Panel This Panel consists of all the different 

about a selected player or team. When the user selects 541 parameters that affect the pitching graphics displayed. The 

"Defense** the user would fix 542 die '^Defense** Panel On sulvPanel in diis Panel would cover parameters ranging 

one side this Panel would allow the user to define ^^^lich ^ ^^^1 time period to pitch location and playing conditions 

player or teams statistics will be displayed, turn on or off die ^^d indude a default s^g. 

^tics and graphics c^ons and set 551 the "Statistics the user has made the desired selections, or 

Parametors sub^Pancl Cto the otiicr side the statistics as changes, the user would sdcct OK and die unit would 

defined by the selections would be displayed 552. ^^^^^ ^ appropriate gr^cs. 

Defensive statistics could consist erf diances, oras, 55 The infannatioo needed fa: the di^jlaycd items, statistics, 

assists, fielding tfirowing arm of pkycx. etc. ff die player balltracks, ct cetera, would be citiici stored in the monary of 

selected is a catcher die number of runnas ttirown out ^^^^ ^ ^ ^ limited memory, 

stealing and %ttirown out would be included in die statistics downloaded via die two way digital radio link from the 

list All statistics would be in die unit memory, or down stadiuradatabase when requested by die user. As soon as die 

loaded as needed from the stadium database. As diffCTcnt eo user had finished using informktion downloaded via die 

players where dt^gnated the Panel would diange to display ^^^^ .^,3 memory. The unit 

die statistics of the designated player. j^^y^ database again if die same infor- 

ff '"Graphics** is turned on, die unit would gr^hically mation was needed by die user, 
represent, in the form of 'domes', the defensive range of the 

selected players according to the selections made in die 65 TEAMS 25^5 

*^aphics Parameters** Panel. The defensive range of a The "Teams** Main Menu Option gives the user access to. 

player is defined by the players physical ability along with and the ability to fix around the stadium, lists of players or 
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team sdiedules. These lists may also be used to designate 
players for other Main Menu (^>tions such as Offense and 
Pitching. If the user selects "Tfeams** the unit would display 
and the user would fix a PaneL the "Team Select" Panel 

If the user selects *X>ther Team" the unit would fix the ^ 
logos of all teams, ordered according to divisiOD and 
standing, as a Panel in front of the user. On one side of the 
Panel the National League would be displayed and on the 
other side the Andean League. The user would then select 
a team, the Logos Panel would be deleted and the user fix 
fiiat teams entire roster as a PaneL If the home or visiting 
team was selected the user would fix the Roster Type Panel 
consisting of Uneup", **Bullpen", ''Bench'* and "Schedule** 
buttons. The user would select the desired list and the 
selected list would be fixed as a Panel by the user If ^ 
Schedule" was selected the teams sdicdulc, including 
infonnation on previous wins and losses, times of games etc. 
wcNild appear as a Panel and be fixed by the user. 

STADIUM 296 20 

The Stadium Main Menu Option gives the user access to 
information about stadium services, ameoities and infra- 
structure and visually displays the information to the user. If 
the us^ selects ''Stadium'* the unit would display the Sra- 
dium Main Panel which would be fixed \^ere desired. This 
Panel would contain all the different categories available to 
the user. 

As. the various category buttons on the Stadium Main 
Panel are designated icons representing the closest locations 30 
in the selected categories, based upon the selections made in 
Stadium Options, would be displayed or deleted throughout 
the stadium. These icons would be displayed cither directly 
over the location, if it is visible, or over the nearest door to 
the actual location of the selection with appiupiiatc arrows/ 3s 
directions. If the user selected the "Stadium Options'* Main 
Menu Option the unit would display, as a Panel to be fixed 
by the user, the "Stadium Options" Pand listing all the 
option in each category. By making the af^opriate selec- 
tions the user can define what Is displayed in die **Stadium** 40 
Main Menu Option. 

FUN! 297 

When the FunI Main Menu Option is activated the user 
would be aslced to fix &e Flint Main Menu PaneL On one 
side the panel would contain tiie interactive sub-panels and 
on the odicr the rcenactmcnt sub-pand. 

lOTERACnVE BETTING AND GAMBLING 

50 

The Interactive" sub^ands allow the user to tr y_jnd 

p redict whg is going tn happen' rtitqnfy thpt pmft^ 

The Real Money sub-Panel gives Ac user the choice of 
betting real money, in states where fins is l^al of course, or 
'^'un Money** by way of a switch set to one or ttie other. The 55 
unit would know if this type of gankbling was l^al in that 
state and if it was not would not display the Real Money 
sQb-Panel, instead automatically setting 'Tun Money** as the 
OHnage for the interactive sub-Panel. If "Fun Money** is 
selected the user is given a set amount of *^n dollars" which 60 
may be wagered in the games. At the end of the game tbe 
three Brig^teyes user at the game with tiie most **fun doUars" 
would win prizes. If real cold hard cash was selected the unit 
would check the users credit via the Transaction Data 
System 4S and inform them of their limit. If there was 65 
insuf&dent credit available the unit would inform the user 
and switch to Fun Money. All losses and gains would be 
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credited or debited to or from the aj^nxipriate Real or Fun 
Money accounts and would be displayed on the screen edge. 

The Bet sub-Pand allows the user to set amount they 
would bet per selection. To set an amount the user would 
'dick* on the different notes until the desired total is reached, 
Le. to bet fifteen dollars the 'five* would be designated three 
times. The amount would remain unaltered until the user 
selected "'Qxange** and tbtn reentered a new amoimt The 
on/off switch in the real money option is to enable the 
gambler to take a break, or practice for a while, and play the 
games without any wagering. Tlie reason for not having this 
switch in Fun Money option is that the user has nothing real 
to lose and everything to gain. 

The *Xjhost Fiddei** sub-Panel gives the user access to the 
first game. To play the user would switch on, set a b^ 
amouist per fielder moved, designate "Select" and then 
designate which fielder or Adders they fed are in the wrong 
position. The user wcmld then position *ghost ioiages* of 
these fielders in the positions on the fidd they desire and 
designate the **OK** button. The unit, or central database, 
would then track the next ball in fair ground. Jf the ball 
passed through the ghosts 'defensive bubble' and not 
through die real players bubble the user would win. If the 
ball passed through the ghosts 'defensive bubble* and 
through the real players bubble the user would lose. If die 
ball passed through the real players bubble and not through 
the ghosts the user would lose, die ball passed through the 
real fiddcrs bubble and also through another tmsdected 
fielders bubble the bet would be off^. If the ball passed 
through neither bubble die bet would be off. After each fair 
ball the ghost images would disappear: This game would 
require tracking of the ball, induding altitude. 

The 'T lck-a-P^di" sub-Pand would give thc^iaayer 
access to a ^same that lets them tr y to i:g ^ ct.whcreJhe^ncx| 
f aiFBalTw ould Ast hit the gr ound oT S'^ ^i yii ereLtfae 
fi elder is eithe r standing or lands. To pl^the player would 
turn on the sidPPan617^t a Set'amount if different from that 
displayed, designate the Pidt-a-Patch button and then pick 
their "patch** by targeting die desired area with the crosshair 
and pressing the designate key. Only one patch may picked 
per play and it would have to tie at least partially in fair 
territory. Once the player is satisfied with the patch they 
have picked they would designate die OK button in the 
assodated PaneL The ''patdi'* would he represented by a 5' 
dian>eter, at field level, red ctrde centered on the sdected 
position as it would be seen relative to the usee The odds 
would increase the further from home plate the patch was. 
If the ball or player who caught the ball was in their potdi 
the player would win. '^"^^'"'^ 

The **Predict Score** sub-Panel allows ttic player to try and 
predict the final outcome of the game, the winning pitcher 
and the play cr diat drives in the winning run. The player may 
bet on dl or some of the <^)tions with the same or different 
amounts as they see fit To selea a final scx^ the user would 
designate the score button. A Panel containing the team 
names and numbers one to nine would be fixed in front of 
the user, the user would designate each team name and score 
and designate "OK**. The sdected score would be entered as 
the prediction. The score prediction could be changed for the 
first three innings, widi lower and lower odds, and at the end 
of the third would be fixed. The names of the winning 
pitcher and i^ayer who drove in the winning run would be 
entered by designating the ^ipropriate button and designat- 
ing the name from rosters fixed in front of the user in the 
normal way. Each playors odds would be different according 
to their seasons performance in the appropriate categories. 
The names could only be put in before the first pitch. If the 
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final scare was predicted befcre the game, and the conea 
two names were also picked, the TVifccta, the user would win 
a substantial prize. 

The Fun Main Menu Option would also contain a rccn- 
actment sub^anei. This sub-panel would contain buttons for 5 
historical events that have taken place in that stadium and 
button s for dififeient groi^nngs of leoiactments. When the 
History bntton is designated the unit would display or delete 
icons around the stadium that mark a historic spot in the 
stadiums history. For exan^xle, there is a histcMy icon located 10 
in the left field buUpen area. The user would place the 
orosshair of their noit over the icon and press the designate 
key. The icon would change to a description of the histcsical 
significance of the spot, a Re-enact" button and a 'Done' 
button. In this example the descdption would read, ^Hank ts 
Aaron's 715th home run to break the Babe Ruth's record. 
Hit on Apr. 8ft. 1974". If flie usei designated the ^Tte^nacT 
button the user would see, from tiieir perspective, a com- 
puter animated version of the actual event Ilie animated 
sequence would be down loaded firom fte Transaction Data 20 
System 4S and the unit would block out all present players 
and replace them with images generated by the computen 
When the re-enactment was started by designating home 
plate the animation sequence would run. coir^lcte with 
audio. They would see Hank Aaron hit that home run, and 25 
it would be seen by them as if they were there, occupying the 
very seat that they were sitting in for the present game. Not 
the same old footage run on television. The user could view 
the action again by designating the **Re-enaclf* button, or 
dose the descrqAion by designating the 'l>one" button. The 30 
descr^tion would disappear and the icon would rcaprpear 
over the historic spot The number and placement of these 
icons would depend on which stadium you wcare at Chica- 
go's Wrigl^ field would have a great deal of histcmc icons 
due to Ihe age if the stadium, tut a fairly new stadinm like 33 
Baltimore's Camden Yards would have relatively few tns- 
toricai icons. 

The re-enact sub-panel would also have several buttons, 
such as the *This Day in Baseball" t)utton, that when 
designated would ask the user to fix a pand containing 40 
related reenactment buttons. If this button was designated 
the unit, knowing what date it was by referring to it's 
internal dock, or querying the Transaction Data System 48, 
would list several historic events that had taken place that 
date. The uso- would designate an event button and a 4S 
'^Description'' Pand for that event would appear on the 
screen. The **Descriirtion'' Pand would be exactiy like die 
historical "Description** Pand with a 'Tteenact" button and 
a "Qose** button. If Ae user chose to re-enact the event die 
database would down load the ccMi^Juter animated sequence 50 
complete with audia The surronndings would gradually 
change, as viewed through the unit, and fiie user would find 
ftonsetves seated in either a totally different stadium, but in 
a seat wift the exact view of the field as their view of the real 
game, or in the same stadium years ago in the same seat. The ss 
playing field, especially the infidd diamond, would still be 
the real surface they were seeing that day, and the animation 
would take place on top of that surface. If fte user were 
seated in a seat that did not exist in the stadium they appear 
to be in they would see the re-CDactment as if ftey were 60 
floating in mid air. If their seat position was located such that 
ftcir ability to see the playing fidd was Uocked« then ftcsc 
obstructions would automatically be removed from the fidd 
of view so fte user's perspective of fte playing fidd would 
remain fte same. The user would watch fte event until fte 65 
sequence was conqiLete and then fte view would morph 
back to the present day view of the stadium. The user could 
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eifter view the sequence again, or dose the description wift 
fte dose button located on fte description Pand There 
would also be ofter buttons that covered other types of plays 
sudi as Homers of the month. Pitching magic and Flying 
fielders that would function exactly the same way as 'This 
day in BascfoalT*. 

Once an option had been designated the unit would fix a 
Pand listing fte individual plays on fte bore sigjbt in firont 
of the user and fte user would designate fte desired play. 
After fte desired play had been designated fte Pand would 
be ddeted and fte unit would fix the "Description" Pand foo: 
ftat play, and re<nact fte designated play ficom fte users 
perspective if desired. 

The Fun Main Menu Pand would also have a "Fantasy 
Flayer" sub-paneL This would allow fte user to make an 
existing player ^:pear to look like a great player from fte 
past, uniform and alL The user could, for instance, make it 
look like WDlic Mays, the "Say Hey Ed", was playing 
center, instead of soma green rookie who had just been 
called up from the minor leagues. It could not make fte 
player play any diffexendy, it would just make him appear, 
only while viewed ftrou^ fte Brightcycs unit, to be some- 
one dse. The user would designate fte "Fantasy Player" 
option and a Panel containing a list of players from fte past 
would be fixed by user The list would indude a button for 
each name. The user would designate a player from fte list 
The crosshairs would now be blinking, and fte user would 
designate fte real player ft^ wished to cover up wift fteir 
Brighteyes. The player designated would be replaced by fte 
player previously chosen from fte list The user could access 
information about fte Fantasy Flayer just as ftey could a 
normal player. To caned fte effect fte user would return to 
the Fantasy Flay ex Panel and switch fte player's name off. 

POST GAME MODE, FIG. 86 

Post Game Mode is active from fte time that fte Post 
Game Button on fte Post Game Pand in Game Mode has 
been designated, and confirmed, until fte time that unit is 
turned off. There are seven Main Menu Options in Post 
Game Mode. The Post Game Mode consists of fte following 
routines: Transport, Box Score, Standings, Next Game Info., 
Stadium, Game Replays, and Location. 

TRANSPORT 
The Transport Main Menu Option gives fte user access to 
directions to fte various meftods of tranqxxrt available at 
that stadium. When fte Main Menu Option is activated, fte 
unit wiU ask fte user to fix the Ikanspcft Pand on bearing 
only. This is so that fte Panel will stay in fte same place in 
relation to fte user when tttc user moves. The user may ask 
for directions to fte stared arrival point (see Pre Game 
Mode), fte nearest taxi stand or, if availaUe, bus stops or 
trains. To receive directions the user sixoply designates fte 
desired sdection. If arrival point or taxi was designated fte 
approfffiate directions would appropnate directions would 
appear on fte ofter side of the Transport Pand, once again 
on bearing only. This Pand would contain a list of available 
buses or trains. When a specific bus or train has been 
designated the directions to that site would be displayed of 
the "back side" of the Transport Panel as before. All direc- 
tions would be from fte last saved location. 

BOXSCX>RE 

The Box Score Main Menu Opti<m allows fte user to fix 
fte box score of fte game as a Pand to be looked at later: 

LANDINGS 

The Standings Main Menu Option allows fte user to fix 
fte current league standings as a Panet National on one side 
and American on fte ofter. 
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NEXT GAME 

The Next Game Main Menu Option Allows the user to fix 
next game infonnatioD as a Panel This informatioD would 
be: location, start tunc, opposing team, and anticipated 
pitchers. 

STADIUM 

The Post-Game Mode Stadium Main Menu Option func- 
tions in exactly the same way as the Game Mode Stadium 
main Menu Option. 

REPLAYS 

The Replays Main Menu Option gives the user access to 
a list of rq>lays from the just completed game. When ftic 
option is activated the user would fix the Replay PaneL This 
Panel would contain the list of replays available. By desig- 
nating the desired rqplay the user would see the play run 
again from the perspective of dictr seat during &e game or, 
in the case of multiple users, from the perspective of the last 
user. Note that the user does not have to be in their seat to 
sec a replay but they must be within range of the Transaction 
Data System 48 radio link. 

LOCAnON 

The Location Main Menu Option allows the user to input 
their location around the stadium. Hie Post Game Mode 
Location Main Menu Option functions exactly as the Pr&- 
Game Mode Location Main Menu Option does. 

Althougli the present invention has been de^bed in 
considerable detail with reference to certain prcfarcd ver- 
sions thereof, other veisioiis are possible. Therefore, the 
spirit and scope of the appended claims should not be limited 
to the description of the prefezred versions presented here. In 
particular; it would be an obvious extension of the invention 
to other sports which arc enjoyed by spectatOTS. 

We claim: 

L An electro-optic vision system for viewing a sporting 
event coniprising: a camera for producing an electronic 
image of a scene being addressed, a position and attitude 
determining means for detennining the position and attitude 
of the camera, a data store responsive to the outputs of said 
position and attitude detennining means and having infor- 
mation stored therein, a conq>uter operable for combining 
the stored information witti the electronic image to produce 
an augmented image, and an electro-optic display operable 
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for displaying the augmented image, said conq>utcr Is 
divided into three conq^utcr sub-systems: a portable com- 
puter, alVansaction Data System (48); and an Action Data 
System (46), the portable computer being remotely located 
5 but in coimmmication with the Transaction Data System 
(48) and in communication with the Action Data System 
(46). 

2. An ^paratus of claim 1 further con^xrising transducers 
in communication with the computer, the transducers being 
xo fixed about the event fomm and directed to make measure- 
ments from objects within the field of interest whereby said 
measurements are converted into information which rdates 
to &e scene being addressed. 

. 3. An apparatus of daim 2, said transducers being oper- 
15 able for TT>r.a<m n'ng the dynamic position of an element from 
the group: players; baseball; coaches; referees; umpires; bat, 
in play. 

4. An apparams of daim 1, said position detennining 
means being further comprised of a plurality of transducers 

20 at fixed locations about the event forum operable for trans^ 
nutting a signal which enables said computer to determine 
the positi<xi of the camera. 

5. An apparatus of claim 1, said position determining 
means being further comprised of a second data store having 

25 prior knowledge of seating locations about an event stadium, 
and a rneans to enter a q>ectator seat number 

6. An apparatus of cMm 1, said data store further com- 
prising recorded data which rqnrcsents events from a pre- 
vious time in a format that can be combined with images of 

30 a real scene being addressed by said camera. 

7. An apparatus of claim 1, said data store further com- 
prising recorded data which represents symbols in a format 
that can be combined with images of a real scene being 
addressed by said camera. 

35 8. An apparatus of daim 1, said data store further com- 
prising recorded data which represents statistical infnnna- 
tion in a f ofmat that can be combined with images of a real 
scene being addressed by said camera. 

9. An apparatus of daim 1, said data store further coiUr 
40 pnsing reccrded data which represents action traces in a 

format that can be combined with images of a real scene 
being addressed by said camera. 

10. An apparams of daim 1, said data store further 
comprising recorded data which represents computer 

45 grq>hical interface devices in a format that can be combined 
with images of a real scene being addressed by said camera. 

♦ * * ♦ ♦ 
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